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ABSTRACT _ _ 

The Transition Program for Refugee Children (TPRC) is 
designed to provide target stud linguistic and culttxral 

skills necessary for a successful transition into the educational and 
societal mainstream. In 1982-83, the program served 885 students in 
18 New York City high schools. TPRC funds-*-provided under the Refugee 

Act—supported classes in English as a Second Language (ESL) or 

content-area instruction with an ESL approach. Because the Refugee 
Act funds were limited ^amounting to six full-time teacher positions 
distributed across the 18 schools^ such monies necessarily, were 
combined with funds from other sources in individual schools. As a 
result, the extent and kind of services actually provided varied from 
site to site. In addition, funds for 1981-82 and 1982-83 were 
combined and were allocated late, this report presents the assessment 
instruments and procedures and the results of testing to evaluate 
student achievement in 1982-83. The data provided suggest that tPRC 
students madeprogress in their knowledge of English syntax and in 
mathematics. It is difficulte however, to offer meaningful 
interpretations of the data fortwo reasons. Firsts only alimited 
number of cases were reported i Second^ _ thG---cbmbi nation of funds to 
create coherent academic pro while justifiable from an_ 

educational standpoint, makes questionable the attribution of growth 
to any individual funding source. (6C) 
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I. nVFRV/IF^i^ 



the Transition Program for Refugee fTitiiliiren (T.P.R.r.) is 
desi<jo*»<! to provide target ^tuHents with ike 1irgufst!c and cultural 
skills necessary for a saecessfui transition into thf» mainstream of the 
school so that they may function in an Pgl is h- speaking society, t.P.R.C. 
funding supports classes in English as a second language (F.5;.t,) or 
cojitent-area instruction with ah E.S.I.. approach, specifically designed 
to improve participatirng students* English language listening, speaking, 
reading, and writing abilities* the prograffi also proposes to acclimate 
sturtents to toericin life and culturei 

Funds provided under the Refugee Act are limited, amouhtihg to 
the equivalent of six fiill-time teacher positions which are distributed 
across the 1ft schools whose students generated the funds iity-wide. 
ThuSi thp typical pgfticipating school may receive f^indihg amouhting to 
ii.d full-time eguivalehtSi to He used to provide instruction, niven the 



1ln!t.e!.i naf.urie of the fondingi Sefo^ee Act nionips are ne-cessarily com- 
tairefll wit*ii funds from other sources In tHe imlivTHnal !icfeQolSi As a 
resaiTt, t»>e extent and kind of services actually provid^i to an entitled 
student nay vary considerably frocn site to site* 

Also Hsvinq an impact On the implementation of services for 
refugee stodents is tHe fact that the 10R1-R2 and 1982-83 allocations 
were ctSnbined into one. In addition, the monies were received and 
allocated late, so that services funded by the Refugee Act were im- 
plef^ented only in the second half of tHe 19fl?-ft3 school year* this 
would appear to make comprehemive planning very difficult in the partic- 
ipating schools. 

The necessary mixing of funds, variations in services, and late 
receipt of funding^ make it mrist difficult to isolat^the effect of 
T.P.R.r. funding for evaluation purposes. As a result, student outcomes 
should be interpreted with caution. 

in lQP?-fl3, 88^ students were served in IP New Vopk Hity high 
schools, table 1 presents the number of program students by site, ind 
Table ? presents a breakdown of the students by grade. 
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TARtF 1 

Nunher of Program Studefits by 5;ite 



School 

F.P.F.E. School 40 

Far l^ockaway 1^ 

Flushing 2fi 

Newtown 1^*^ 

tohq Island f:ity fi* 

Franlciih Pi Roosevelt 3*5 

Erasmus Hall 24 

New Utrecht fi9 

Ahraham tincbln ^2 

Midwnod ?3 

John F. Kennedy 30 

Theodore Roosevelt 101 
V*a1ton 

Christopher rnlumhus 3? 

Park West 70 

Lower Fast Side Prep l:^ 

Washihqtdh Irvi^ntj ?0 
Seward Par^ 

TOTAL 



5£urce^i Project records: roster of students whc^ 
generated the funds i» 
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tABLE 2 

Ntfnber af Prdgran Students by ra-arie 





Nunher of Students 


q 


?1S 


in 


3fi3 


11 




1? 


ins 


TOTAL 





*Rrade information was missing for 54 stodentSi 
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II. FlImiNGS 



this sect i OR presents the assessment ihstrun^rits arid procedures, 
and the results of the testing to evaiuate student achlevciiseflt in I9S2- 
83. The data set for the Transition Progrsn for Refugee Children was 
assCTbled in the following raaaner. The Divisibh of Hip Schools supplied 
a roster of students served. Sose of these »ere served in ^hools 
receiving fahds and participating in the Chapter I/P*S.E.N. E.S.L, 
prograti. these students were tested with the CREST as part of the on- 
going evalaatibn of that progran* Other t.P.R.C. students were served 
in schools not receiving Chapter I or P,SiEiNi funds i these students 
were CREst tested by the personnel of the Division of High Schools and 
approximately 399 data sheets for thesi students were submitted for key- 
puhching along with those from the Chapter I/PiSiE.N^ prOgrsn to foriii 
one data set* the roster of students served with T.P.R.C. funds *«s then 
matched against the CSJEST data set and scores for T.P.R.C. students sere 
arawh from the data set for analysis. In the interest of cOTpleteness» 
another match was performed to extract scores on the New York Ci ty Reading 
antf Mathematlcs^sts^ for those students w*ib lacked CREST scores, 

fm r©suUant nw^ber of cases with complete test scores was 
ntMr low for several possible feasant* The number of data ifeiets lab- 
mi tted for non-Chapter I students at « particular scHboli for example, 
sorw»times differed si;ft)Stantlai ly frm the number of students at that 
school who generated the fuhdSc Errors fti data entry generally account 
for some loss of infinnation especially when files are matched, Addi^ 
tibnallys It appears that ftany students, although in E.S.t, cJasses, did 
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not have test data reported. The possible riasdhs for this are many: 
sisne students may net have been CREST tested because of absehee* fet 
others might hot have scores on city-Wide tests due to absence on testing 
dates OF because they ^re excused frdri the tests doe to their recent 
Immigrant status. Because subject attrition may not have been ah entirely 
randSn process, the resulting subset of stuiehts for whew cceiplete data 
are available may rnot be representative ef the prdgrSn population. 



to measure English language achlevemeht. The CREST was developed by the 
New York City Public Schools to rneasure mastery of instructional objectives 
of the E.S.L. Curricula at the high school level. There are four itOTS 
for each objective-j and mastery of an objective Is achieved when three 
of the iteiifls are answered correctly. The test has th^ee levels: beglnhlng 
(I), intermediate (Ilj* and advanced (III). The maximum score on levels 
I and I! is 25, while the tiiaximofn score on level III is 15. 

Mean differences between pre-test and post-te-st are calculated 
to represent the gain score* and an index which represents the number of 
objectives mastered per month is cbmpotedi However^ since the levels are 
not equated wMrtVctU^ * it is iiRpossible to measure gains for students 
who change ItvelSi Extensive information ofn CREST objectives and psycho- 
metric properties appears in the Tgcftntcal H^fiaal^^efcL Vork City English 
AJ^LjSee^ohd Lahquaqi Criterion Referenced English Syntax^ st. 



*B6ar<i of Edycatiort of tf?s City of New York, 6iv1s1o«i of High Schools, 




The Criterion Refe^*! 




(CREST) was used 



1978. 



-6- 

11 



The HREST was aelihihi stered to bnth the Chapter I and ^en^Chapter 
I qroups at the beginning and end of the spring semester. Hata mre 
niissinq or incomplete for approximately half of the students. 

Table 3 presents the test results for Chapter I students who 
were pre- and post-tested with the same test level during the semester. 
Examination of Table 3 reveals that Chapter I students mastered an 
average of Uld objectives per month in the spring termi City-wide 
objectives for E.S.L. students are for the acquisition of one CREST 
objective for every four weelcs ef school attendance. As a whole^ Level 
I and II students realized the city-wide standards, hevel HI students 
qained an average of n^BR objectives per month. There were slight 
celling effects manifest on all three CREST levels, which were stronger 
on tevei III and would thus have lowered the observed gain fnr students 
tested on that level . 

Results for the non-Chapter I students are presented in fable 4. 
There wa$ a total of cases with valid pre-test and post-test scores 
on Levels I through III. Average gains per month were 1.33 CREST objec- 
tives for students tested on Level I, 1.?^ CREST objectives for bevel 
n students, and n,^R CREST objectives for students at Level III. 
Ceiling effects were observed on all three test levels but were not 
sufficiently strong to negate gains in English comprehension made by 
thes^ students, nveralli these students surpassed the city^wide accepted 
criterion of one CRFSt Object ive mastered per menthi 
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TABLE 3 



Results of the £H-t-erie)ri Referenced English Syntax Test 



Spring 

Average Number of Average Objectives 

Test Number of Objectives Mastered Objectives Months of Mastered 

Level Students Pre Post Mastered* Treatment Per Month 

i 64 14.17 18.67 4.50 3.06 1.50 

11 84 16.17 20.14 3.98 3.87 1.30 

III 82 10.43 12.21 1.78 3.01 0^ 

TOTAL 230 13.57 16.90 3.34 3.95 1.10 



*Post-test minus pre-test. 
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TABLE 4 

Results of the Eriten'dh Referenced English Syntax Test 
(Non-Chapter I Students Pre- and Post-Tested on Same Test Level) 









Spring 








Test 
Level 


Nujfibgr of 
Students 


Average Number of 
Objectives Mastered 
Pre Post 


Objectives 
Mastered* 


Average 
Months of 
Treatment 


Objectives 
Mastered 
Per Month 


I 


51 


16.20 


20.15 


3.96 


3.08 


1.33 


IE 


94 


16.61 


20.40 


3.80 


3.14 


1.24 


HI 


m 




12.69 


1.99 


3.11 


CW5 


TOTAL 


225 


14.41 


17.60 


3.19 


3.12 


1.05 



* 

Post-test minus pre-test* 
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Another instrument used to measure achievement in reading and 
writing iti English was the New York City Re ading Test , actually two 
standardized tests which have been renormed with students from the New 
York City public schools. Grades 9 through 1?? use the fjamprehehsive 
Tast^f Bas ic Skills {C.T.R.S.| and the Calif ornia j^cMa ^temen^ Test 
(CAT) as follows: 

Grade Test Level 

9 r.T.R.S. 3 

in CAT la 

11 C.T.B.S. 4 

1J> CAT 19 

Information bh psychometric properties may be obtained from 
the test publishers. New York City norms may be obtained from the New 
York City Public Schools, nffice of Testing, lin tivinqstbn Street, 
Brooklyn, New York llRni. 

"Fffect size" was calcula.ted for each grade leVel * following 
the procedure recommended by Cohen.* An effect size for correlated t^-test 
is an estimate in standard defviatidns, freed of sample size, of the 
difference between means. Effect size provides additional substance to 
the analysis as it may be interpreted in light of Cohen's reconwehdations: 

i2n = small effect size 

.Sn = moderate effect size 

.«n = large effect size 
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fahie S presents results for the New Y ork CMy Reading Tpst i 
(Inly small numbers of stiiderits were reported ^ possibly reflecting the 
fact that most T.P.R.C. students are recent arrivals, and might have 
been excused from city-wide testing. Results indicate that raw score 
gains were statistically significant for the tenth and twelfth graders 
with moderate to lar*ie effect sizes for these students. f^iritH graders * 
although manifesting a small positive ^rfeet sizs^ remained at the 
eleventh percentile on both pre- and post-tests. Tenth graders rose 
from the hinteenth to the thirty-first percentile, for the largest gain. 
The fact that ninth-grade students did not perform at a higher post-test 
percentile than their pre-test indicates that perhaps the test was too 
difficult for them and that another test might be used with these students 
of limited Fnglish proficiency. 
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TilE 5 

Englisfi UUm Achievenient 

Significance of the Total Raw Score Differences Between initial and Final Test Scores of 
Students with Full Instructional Treatment on the New Yoric £it jLBeadin(j Test ^ l>y Grade 



Hean 



Srade 


N 


Pre-Test 
Heart Deviation 


Post-Test 
Mean Deviation 


Hean 

Difference 


Corr. 

Pre/post 


T. 

Test 


Level of 
Significanci 


Square 
i Hi thin 


Effect 
Size 


(i 


9 


28.S6 


6.25 


39.67 


7.75 


2.11 


.30 


0.76 


.479 


8.34 


.25 


10 


17 


29.71 


12.39 


35.00 


13.64 


5.29 


*79 


2.59 


.020 


8.43 


M 


li 


* 






















n 


16 


29.56 


8.59 


35,81 


9.47 


6.25 


.67 


3.41 


.004 


7.34 


.85 



'insufficient number of students to perform statistical test. 
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HatHefTiat.lcs achievement was measured w1t»i the few Yanrfe Tity 
Wathpmatif.l^ Tgst ^ The New York City Math Test, employs the Stanford fi&st 
of Academic .^idlXi (TASK) and is designed to measure general mathematics 
competence. It emphasizes arithmetic and numeric concepts and applica- 
tions witfi minor emphasis bh algehras geometry^ and measurement. The 
TASK, which has two forms i is a two-level test. Level I is designed for 
grades R, and 10; hevel II for grades U and 1? and junior college 
level. The TASK was normed on two national samples. Further information 
is available from the Psychological Corporation, 75nh Old Oak Blvd., 
Cleveland, nhio 44130. 

Results of this test are presented in Table 6i Refugee Program 
students showed statistically significant raw score gains for all grades 
and moderate to large effect sizes. When these results were converted 
to percentile standings, gains averaged six fsercehtile points for each 
year. Post-t«»st percentile ratings were 3? for both ninth and tenth 
graders and stood at the sixty-sixth percentile for the twelfth graders. 



ERIC 



-13- 

Id 



TABtE 6 



HatheiiHtics ftchievement 

Siqnifieafice of ttie Total Raw Score Differences Between initial and Final Test Scores of 
Students wltti full Instractional Treatment on the Hew York City Hatheinatics Test , by Grade 



(>rdde 


fi 


Pre-Test 
Hean Deviation 


Post-Test 

Mean Beviatldn 


Held 

Bifferenee 


Corr, 
Pre/post 


T- 

Test 


leyel of 
Sipificance 


Effect 
Size 


9 




25.15 


9.43 


28.70 


9.77 


3.56 


.M 


2.64 


,014 


.51 


10 


82 


29.82 


9.10 


31.93 


7.94 


2.90 


*72 


4*06 


.0001 


.45 


11 


51 


21.18 


?,89 


14.98 


8.22 


3.88 




5.93 


,0001 


.83 


12 


50 


31.64 


8.07 


34.14 


8.61 


2.50 


.72 


2.82 


.007 


.40 



29 
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in. rnrrLii*?!n"N5; hm p;FrnMMrNnATinri<; 

The availahle (\<\U suqqest that f.P.P.C, students are progressing 
in their knowledqe of Friqlis'h syntax and in "lat hematics. Mevertheless * 
aUhouqh Refuqpe Proqram students met New York City quidelines for Fnqlish 
lanquaqe achieveftient , cnmplete TRFST data were available for only 
percent of the participatihq students. Infortnation was also available 
for only &? additional students tested with the fiew Vork City Reading Test . 

The sources of the difficulty in collecting information bh 
T.P.R.C. students city-wide are many, and havi b«en indicated in the 
intrbductdry Section of this report. It appears that information for 
many students is not being processed centrally (I.e., hot all students 
nay he in E.S.L. classes, or r.R£ST-tested). in addition, clerical 
errors accumulate as data are subjected to successive manipulations, 
resuUlhq In lowered match rates across data files. In respbhse, future 
data collection efforts will attempt tb "flag" centrally-processed 
records tb better control the infiormatlbh now lost thrbugh matching of 
records, nther more ambitious resr-onses, such as a city-wide testing 
program for reimbursably funded F.S.t. students * would not seem feasible 
given the variety of treatments the sti"^'"*ts receive, and the excessive 
burden of additional testihq this would impose on the participating schools 
The number of Students with missing or incbmplete Scores * however, suggests 
that efforts should be marie in the participating schools tb test all 
Students and report the Information as ebnpletely as possible. 
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In cbncloslbh. It Is difficult to offer fi^ai^fngful Interpreta- 
tranz of the data for T.P^R.f:. students for two reasons. Clearly^ tMe 
limited humher of cases reported indicates that general izat ions abOMt 
the proqress of the whole qroup of students should be riarie with caution^ 
at best. In addition, however^ the co^^hinatior^ of funds to er^^ate 
coherent academic programs, which is educationjil ly correct and sensible, 
makes the attribution of qrowth to an Individaal fuhdinq source question- 
abUi ft better approach to the evaluation of the progress made by 
T.P.P.r. students would be to conduct a global assessment ^f the delivery 
of services to LFP students across funding sourr:es. 

A f^hal Issue of importance involves the Implementation of 
services supported by thie Refugee Act, Funding ^as received at mid-year^ 
malcihg it difficult to plan for the optimal use of the monies. This 
might have resulted, for example ^ in programs not being able to Identify 
and place staff in the falU posstbly resulting In less systematic or 
extensive services during the year due to the lack of personnel, nearly^ 
timely provtisioh of Fuhdinq would i1T^p^dve the planning and implementation 
of services to T.P.R.C. students. 
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